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The  evaluation  and  cleanup  of  current  and  former  military  sites  contaminated  with 
buried  munitions  relies  on  two  well-understood  geophysical  technologies  to  detect 
the  munitions:  magnetometry  and  electromagnetic  (EM)  induction.  As  these 
technologies  were  introduced  in  munitions  response  projects,  the  Geophysical 
Prove  Out  (GPO)  was  developed  to  determine  whether  the  geophysical  data 
collected  would  meet  project  objectives.  Over  the  last  15  years,  numerous  GPOs 
have  been  performed  on  a  variety  of  site  conditions,  and  a  significant  body  of 
knowledge  has  accumulated  documenting  the  performance  of  these  technologies. 
This  accumulated  understanding,  along  with  the  recognition  that  magnetic  and  EM 
responses  of  munitions  may  be  predicted  reliably  using  physical  models,  presents 
the  opportunity  for  both  streamlining  and  enhancing  the  GPO  with  a  more  rigorous 
physics-based  approach. 

A  Geophysical  System  Verification  (GSV)  process  is  envisioned  in  which  the 
resources  traditionally  devoted  to  a  GPO  are  reallocated  to  support  simplified,  but 
more  rigorous,  verification  that  a  geophysical  system  is  operating  properly,  as  well 
as  ongoing  monitoring  of  production  work.  Two  main  elements  are  considered  in 
this  course: 

Instrument  Verification  Strip  and  Blind  Seeding  Program. 
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Analog  Sweeps 


Historically,  clean  ups  were  addressed  using  analog  sensors  such  as  shown  in  the 
slide.  Explosive  Ordnance  Disposal  (EOD)  technicians  would  walk  side  by  side 
scanning  the  ground  and  listen  for  tone  changes  to  identify  ferrous  items.  Many 
problems  were  identified  using  this  technique  such  as  no  record  of  what  was  picked 
as  a  target,  what  areas  were  and  were  not  covered  during  the  sweep,  and  other 
aspects  that  leave  great  uncertainty. 
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Early  MR  Projects 


•  No  quantitative  measure  of  what  is  detected  and  how 
deep 

•  Inconsistent  results  from  site-to-site  with  the  same 
equipment 

•  Inconsistent  results  on  a  given  site  with  different  teams 
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During  early  attempts  to  use  detection  technologies  in  munitions  response  projects, 
it  became  obvious  that  much  remained  to  be  learned.  The  first  projects  often  began 
with  a  “test  off’  of  multiple  sensor  types  for  each  site.  Detection  technologies  that 
worked  well  in  one  place  seemed  not  to  work  at  all  at  another.  Results  achieved  by 
one  detection  team  at  a  given  location  could  not  always  be  duplicated  by  another 
detection  team  at  the  same  location,  using  the  same  equipment.  Most  importantly, 
there  was  little  information  regarding  what  these  first  munitions  responses  were 
actually  achieving:  No  one  could  say  with  much  certainty  how  deeply  or  effectively 
the  instruments  were  detecting  munitions,  nor  what  was  being  left  behind.  In  other 
words,  there  was  no  measured,  quantitative  method  of  describing  the  results  of  a 
munitions  response. 

As  a  result,  DoD  managers  and  State  and  Federal  Regulators  providing  oversight  to 
projects  began  looking  for  ways  to  quantitatively  understand,  measure,  and  describe 
the  effectiveness  of  technologies  used  at  UXO  sites. 
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Early  GPO 
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Objective: 

♦  Test  off  to  select  technologies 

♦  Site-specific  capabilities  and 
limitations 

Method:  Empirical 

♦  Bury  items  at  known  location, 
depth  and  orientation 

♦  Score  on  how  many  are  detected 
Outcome:  Understanding  of 
common  sensors 


X  Mortar 
+  Halfshell 
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Munitions  response  projects  began  to  use  GPOs  in  an  attempt  to  determine  the 
capabilities  and  limitations  of  geophysical  systems  under  controlled  conditions  near 
the  work  site.  In  a  GPO,  a  known  number  of  inert  munitions,  surrogates,  and  other 
objects  are  buried  at  precisely  known  locations  and  depths,  and  then  the  site  is 
mapped  with  one  or  more  geophysical  instruments.  The  data  are  processed  and 
targets  selected  based  on  some  predetermined  criteria,  and  the  resulting 
geophysical  map  is  used  to  display  “anomalies”  that  represent  potential  munitions. 
Performance  on  the  GPO  is  scored  primarily  based  on  the  fraction  of  the  emplaced 
targets  that  are  associated  with  geophysical  detections,  often  accompanied  by 
many  other  secondary  metrics.  Over  the  years,  the  accumulation  of  empirical 
results  from  GPOs  resulted  in  an  understanding  and  documentation  of  sensor 
capabilities. 
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GPO  Limitations 
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•  No  information  on  quality  of  field  work 

♦  Limited  representativeness 

♦  Up-front  evaluation  only 

♦  Small  area  allows  for  more  careful  work  to  "pass" 

♦  'A  team'  versus  the  'C  team' 

•  Statistical  uncertainties 

•  Cost  and  time 
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Although  useful,  GPOs  have  a  number  of  limitations: 

Representativeness:  An  inherent  assumption  regarding  GPOs  is  that  conclusions  drawn  from  performance  in 
the  GPO  are  applicable  to  work  on  the  “live”  production  site.  This  requires  that  the  population  of  targets 
and  clutter  placed  in  a  GPO  is  representative  of  the  actual  target  and  clutter  population  that  will  be 
encountered  in  the  field  and  the  geophysical  systems  and  processes  employed  at  a  GPO  sufficiently 
resemble  the  geophysical  systems  and  processes  employed  in  the  field. 

In  fact,  the  true  distribution  of  targets  and  depths  is  unknown  at  the  time  the  GPO  is  constructed  and  the 

resources  available  typically  constrain  the  size  of  the  GPO  and  number  of  targets  that  are  emplaced.  Many 
live  sites  are  vastly  heterogeneous  across  a  wide  range  of  variables,  including  target  density,  clutter 
environment,  geology  and  other  noise  sources.  In  practice,  there  is  almost  always  significant  discrepancy 
between  the  GPO  and  the  live  site  it  models. 

In  addition,  the  GPO  survey  likely  represents  ideal  performance  in  many  applications.  There  are  two  common 
themes  in  the  concerns  have  been  raised  over  the  years  about  how  well  performance  on  the  GPO 
represents  performance  throughout  the  production  work.  First,  often  projects  employ  multiple  crews  with 
multiple  instruments  in  production  work  and  it  is  rare  for  all  teams  and  equipment  to  qualify  each  day.  If  one 
team  passes  the  GPO  at  the  beginning  of  the  project,  no  information  is  available  on  how  that  team 
performs  weeks  or  months  into  the  project,  nor  on  how  other  teams  perform.  Second,  the  GPO  is  a  small 
area  where  it  is  possible  that  the  crew,  motivated  to  “pass”  the  GPO,  can  be  more  careful  than  they  might 
be  throughout  the  site.  Over  time  the  GPO  will  become  familiar  to  the  crews,  so  any  blind  testing  aspect  is 
lost.  Overall,  the  GPO  provides  little  information  on  performance  of  the  system  or  the  crew  while  collecting 
production  data. 

Statistical  Uncertainties:  the  primary  measure  of  success  for  most  GPOs  has  been  the  fraction  of  targets 
detected,  which  was  treated  as  a  surrogate  for  the  probability  of  detection  that  a  system  would  achieve  on 
the  site.  In  fact,  few  if  any,  GPOs  contained  a  sufficient  number  of  targets  to  calculate  statistical  metrics 
with  the  desired  uncertainties. 

Cost:  GPOs  have  ranged  in  size  and  complexity,  from  a  few  targets  to  hundreds  of  targets.  The  construction 
and  performance  of  the  GPO  have  been  substantial  costs  on  many  sites.  Cost  factors  include  identifying 
and  clearing  a  GPO  site,  acquiring  and  seeding  targets,  data  acquisition,  processing  and  report  writing.  In 
many  cases,  the  GPO  has  required  a  separate  deployment  in  order  to  obtain  approval  to  proceed. 
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While  the  use  of  digital  technologies  brought  great  enhancements  to  the  munitions 
response  program,  data  could  easily  be  provided  that  often  left  an  inaccurate 
impression  of  the  quality  of  the  work.  This  chart  shows  data  as  gridded  and 
presented  by  the  contractor.  The  map  indicates  project  requirements  were  met  and 
appears  to  be  complete  and  of  high  quality.  When  the  actual  survey  data  tracks  are 
added  and  processed  according  to  the  project  requirements,  it  is  evident  that 
several  large  data  gaps  were  obscured  in  the  data  presentation.  The  separate 
function  of  performing  GPOs  and  then  moving  into  production  work  created  what 
some  have  commented  as  providing  an  opportunity  to  have  the  "A"  team  do  the 
test,  i.e.,  the  GPO  and  then  letting  the  "C"  team  to  the  work. 
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When  the  actual  survey  data  tracks  are  added  and  processed  according  to  the 
project  requirements,  it  is  evident  that  several  large  data  gaps  were  obscured  in  the 
data  presentation.  The  separate  function  of  performing  GPOs  and  then  moving  into 
production  work  created  what  some  have  commented  as  providing  an  opportunity  to 
have  the  "A"  team  do  the  test,  i.e.,  the  GPO  and  then  letting  the  "D  "  team  to  the 
work. 
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Why  change? 

Continuum  of  information  throughout  project 
Rigorous,  physics-based,  quantitative 
Standard,  repeatable,  comparable 
Better  use  of  resources 


Introduction 


The  GPO  has  served  its  purpose.  There  now  have  been  tests  of  many  geophysical 
systems  at  multiple  sites  over  a  period  of  about  15  years,  resulting  in  a  vast 
increase  in  the  understanding  of  the  capabilities  and  limitations  of  common 
geophysical  detection  systems.  This  understanding,  coupled  with  reliable  models 
for  the  signals  expected  from  common  sensors,  presents  and  opportunity  to  move 
beyond  the  empirical  GPO  to  a  more  physics-based  verification  of  sensor 
performance. 
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GSV  Concept 
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A  Geophysical  System  Verification  (GSV)  process  is  envisioned  in  which  the 
resources  traditionally  devoted  to  a  GPO  are  reallocated  to  support  simplified,  but 
more  rigorous,  verification  that  a  geophysical  system  is  operating  properly,  as  well 
as  ongoing  monitoring  of  production  work.  The  GSV  will  add  to  existing  QC  and  QA 
processes  and  is  not  intended  to  replace  current  quality  practices. 
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GSV  Concept 


Geophysical  System  Verification 


Instrument 

Verification 

Strip 


Production  Area 
Blind  Seeds 
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Two  main  elements  are  considered  in  this  course: 

Instrument  Verification  Strip  (IVS):  The  traditional  GPO,  which  consists  of  several 
tens  to  a  hundred  or  more  targets,  would  be  replaced  by  an  IVS  containing  a 
handful  of  targets.  The  objective  of  the  IVS  would  be  to  verify  that  the  geophysical 
detection  system  is  operating  properly. 

Blind  Seeding  Program:  The  production  site  would  be  seeded  with  targets  at 
surveyed  locations  that  are  blind  to  the  data  collection  and  processing  teams.  The 
objective  of  the  seed  program  would  be  to  provide  ongoing  monitoring  of  the  quality 
of  the  geophysical  data  collection  and  target  selection  process  as  it  is  performed  in 
the  production  survey  throughout  the  project. 
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Physics  Basis  for  Geophysical  System 
Verification  (GSV) 

Tom  Bell 
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Outline 


•  Basis  of  Geophysical  System  Verification  (GSV) 

•  Geophysical  Sensors 

•  Electromagnetic  sensors  (EM61) 

♦  Electromagnetic  induction  (EMI)  fundamentals 

♦  Signal  response  curves 

♦  Industry  Standard  Objects 

♦  Noise  and  signal  detectability 

♦  Measurement  uncertainty  and  error  bars 

•  Magnetometers 

•  Summary 
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Basis  of  GSV 
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•  Well  characterized  sensor  with  predictable  response 

♦  Standard  magnetometer  and  EM61 

♦  Emerging  EM  sensors  to  be  documented 

•  Well  characterized  target 

♦  Common  munitions 

♦  Industry  Standard  Object 


Physics  Basis 
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Well-characterized  sensor:  The  premise  of  this  approach  is  that  the  basic  physics  of  the  sensor  system  is  well 
characterized  and  documented.  Magnetic  signatures  of  items  do  not  vary  from  one  sensor  to  another,  although 
how  the  magnetic  field  is  measured  differs.  Prediction  of  EM  signatures  requires  knowledge  of  salient  features 
of  the  sensors,  such  as  the  transmit  moment,  coil  geometry,  receive  time  gates,  and  so  forth,  which  are  well 
documented  for  the  commonly  used  EM61-MK2.  The  GSV  may,  in  principle,  be  applied  to  any  variation  on  EM 
technology  that  is  transparent  and  documented. 

GSV  will  not  be  applicable  to  so-called  “black  boxes.”  This  will  include  proprietary  devices  for  which  sensor 
details  are  not  divulged  and  any  other  system  whose  operation,  in  terms  of  both  hardware  and  processing,  is 
not  well-documented.  Nor  will  it  be  appropriate  for  technologies  based  on  completely  different  physical 
phenomena,  where  a  GPO  may  be  required. 

Well  characterized  test  objects:  EM  and  magnetic  signals  are  site  invariant  and  any  well  characterized  object 
may  be  used  for  GSV.  Test  objects  may  include  the  munitions  of  interest,  but  that  is  not  essential  for  confirming 
that  the  system  is  operating  properly.  As  an  alternative,  we  introduce  the  concept  of  an  “industry  standard 
object  (ISO).”  We  have  selected  three  sizes  of  commonly  available  pipe  sections  that  can  be  readily  obtained 
from  any  plumbing  or  hardware  supplier.  While  munitions  may  vary  by  make  and  model  number  (i.e.,  there  are 
many  different  types  of  60-mm  mortars),  ISOs  have  the  advantage  that  they  will  be  made  to  the  same 
specification  regardless  of  where  they  are  obtained.  Together,  the  three  sizes  should  meet  the  objectives  of 
most  MR  projects  in  that  the  physics  characteristics  of  one  or  more  of  the  ISOs  will  be  sufficiently  similar  to  the 
targets  of  interest  that  they  can  be  used  to  verify  that  the  system  is  operating  properly  and  can  be  expected  to 
detect  the  targets  of  interest. 

The  ISOs  have  been  modeled  and  measured  so  that  they  can  serve  as  well-characterized  test  strip  targets. 

(Ref.  3)  Similar  data  are  provided  for  commonly  encountered  munitions.  (Ref.  4)  The  data  are  included  in  the 
CD  that  accompanies  the  course  materials  to  allow  application  to  single-sensor  EM61-MK2  systems.  A  simple 
windows  application  is  provided  that  can  be  used  to  produce  sensor  responses  for  any  array  arrangement  of  an 
EM61 . 
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Geophysical  Sensors 


The  workings  of  modem  geophysical  sensors  for  munitions 
response  applications  follow  well  defined  physical 
principles  and  their  performance  can  be  quantitatively 
characterized  with  a  high  degree  of  accuracy  &  precision 


Physics  Basis  4 


There  are  two  basic  types  of  geophysical  sensors  used  in  munitions  response  work: 
electromagnetic  induction  (EMI)  sensors  such  as  the  EM61,  shown  here  in  the 
standard  man-portable  configuration  (left)  and  in  a  towed  array  configuration 
(center),  and  total  field  magnetometers  (right).  A  physics-based  approach  to 
geophysical  system  verification  is  possible  because  these  sensors  obey  well- 
defined  basic  physical  principles,  and  their  performance  can  be  quantitatively 
characterized  with  a  high  degree  of  accuracy  and  precision. 
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EMI  Physics 
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Physics  Basis 
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This  sequence  of  slides  shows  the  fundamental  concepts  involved  in  EM  measurements,  starting 
with  a  picture  of  a  typical  EM  sensor  being  pulled  across  a  field. 
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EM  Fundamentals 


Typical  Electromagnetic  Induction  Sensor 


The  basic  elements  of  an  EM  sensor  are  a  transmit  coil  and  a  receive  coil  shown  by 
the  loops  above  the  ground  surface.  A  current  pulse  running  through  the  transmit 
coil  creates  the  primary  EM  field,  illustrated  by  the  arrows  flowing  along  field  lines 
shown  in  red.  This  pulse  excites  the  munitions  item  under  the  sensor. 


6 


EM  Fundamentals 


Typical  Electromagnetic  Induction  Sensor 


Changes  in  the  primary  field  set  up  eddy  currents  in  the  object,  shown  schematically 
by  the  green  arrows  seeming  to  flow  around  the  buried  munitions  item. 
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EM  Fundamentals 


Typical  Electromagnetic  Induction  Sensor 


Receive  Coil  * 


►Transmit  Coil 


Primary  Field 


Induced  Target  Response 


Physics  Basis 


Ground  Sur fac e 
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The  eddy  currents  produce  a  secondary  or  induced  EM  field  emanating  from  the 
object.  This  field  can  be  represented  by  an  induced  dipole  at  the  object's  location. 
The  strength  and  orientation  of  the  dipole  moment  are  determined  by  the  primary 
field  at  the  object  and  physical  properties  of  the  object  such  as  its  size  and  shape, 
as  well  as  its  orientation. 
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EM  Fundamentals 


Typical  Electromagnetic  Induction  Sensor 


Induced  Flux 
Receive  Coil 


►Transmit  Coil 


Physics  Basis 


Ground  Sur fac e 


5d 


The  induced  field  is  measured  by  the  receive  coil,  the  output  signal  being 
proportional  to  the  rate  of  change  of  the  EM  flux  through  the  receive  coil. 
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Time  Domain  Electromagnetic  Induction 


1)  Target  Magnetization 

2)  Eddy  Currents  Created 

3)  Eddy  Currents  Decay 


Transmit  Current 


Time 
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There  are  two  basic  types  of  EM  sensor:  continuous  wave  (frequency  domain),  and 
pulsed  wave  (time  domain).  Frequency  domain  sensors  transmit  a  continuous 
waveform,  while  time  domain  sensors  transmit  a  sequence  of  EM  pulses.  The 
pulsed  sensor  is  the  most  commonly  used  configuration  because  it  allows  the  eddy 
current  response  to  be  measured  when  the  primary  field  is  not  changing  and  is  no 
longer  overwhelming  the  signal  due  to  the  induced  field.  The  two  plots  show  typical 
transmit  and  receive  waveforms  for  a  pulsed  EM  sensor  and  identify  the  three 
stages  of  the  EM  measurement  process.  (1)  The  object  is  magnetized  only  during 
the  transmit  pulse.  (2)  The  eddy  currents  are  excited  in  the  target  when  the  pulse 
abruptly  ends.  (3)  The  EM  response  is  measured  during  the  eddy  current  decay 
after  the  primary  field  pulse  ends.  This  measured  decay  contains  the  information 
that  is  used  to  classify  the  target. 
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EM61-MK2  Sensor 


SERDP 


The  standard  EM61  averages  response 
over  four  time  windows  or  gates  following 
the  primary  field  cutoff 


Physics  Basis 


1C-1  10D  TO1 

Time  (msec) 


We  will  focus  on  the  standard  EM61-MK2  sensor,  which  is  shown  in  the  field  in  the 
inset  picture.  The  EM61-MK2  samples  the  eddy  current  decay  over  four  windows  or 
time  gates  following  the  primary  field  cutoff.  Measurements  can  be  taken  at  a  rate  of 
about  ten  times  per  second,  corresponding  to  a  spacing  along  a  survey  line  of  about 
10-12  cm  at  normal  walking  speeds. 
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EM  Signal  Calculations 


Dipole  response  model  decomposes  signal  calculation  into 
terms  that  depend  only  on 

♦  Sensor  properties  and  sensor/target  geometry 

♦  Target  size,  shape  and  composition 

V(t)  =  (i0nRnxI0CR*CTB(t) 

^  v  ^  v - V - 

EM  signal  sensor/geometry  target 

Eigenvalues  of  target  polarizability  tensor  B  (response 
coefficients  for  three  principal  axes)  are  determined  from 
controlled  pit  or  test  stand  measurements. 

Physics  Basis  7 


We  can  use  rather  simple  mathematical  formulae  to  represent  the  response  of  the 
EMI  sensor  to  a  buried  munitions  or  clutter  item.  When  the  distance  between  the 
sensor  and  the  object  is  somewhat  larger  than  the  size  of  the  object  we  can  use  a 
simple  "far  field"  approximation  referred  to  as  the  dipole  response  model.  This 
model  splits  the  response  into  products  of  terms  that  depend  on  properties  of  the 
sensor  and  the  sensor/target  geometry  and  terms  that  depend  on  basic  properties 
of  the  target  such  as  its  size,  shape  and  material  composition.  We  refer  to  the 
sensor  and  configuration  dependent  factors  as  extrinsic  factors  because  they  are 
external  to  the  target,  and  the  basic  properties  of  the  target  as  intrinsic  factors 
because  they  are  inherent  to  the  target.  The  effect  of  the  extrinsic  factors  on  the 
sensor  response  is  the  same  for  all  targets,  while  the  contribution  of  the  intrinsic 
factors  to  the  response  depends  only  on  what  the  target  is,  not  where  it  is  or  how 
the  sensor  is  being  used.  Once  we  have  determined  the  intrinsic  response  terms  for 
an  object  (e.g.  a  60mm  mortar),  we  can  easily  calculate  what  the  sensor  response 
will  be  for  any  target  depth,  orientation,  or  location  relative  to  a  survey  line. 
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Sensor  Response  Curves 


•  Target-specific  signal  vs.  depth  and  orientation 


/  7 

0 

•  Measure  response 

•  Strongest  signal  from 
vertical  orientation 

Physics  Basis 


Depth  (m) 


This  sequence  of  slides  shows  how  sensor  response  curves  are  created.  Signal  vs. 
depth  response  curves  for  least  favorable  (aligned  perpendicular  to  the  primary 
field)  and  most  favorable  (aligned  parallel  to  the  primary  field)  target  orientations 
can  be  scaled  to  a  few  measurements.  The  curves  bound  the  range  of  signals  that 
the  target  can  produce. 

First,  the  response  is  measured  for  some  target  depth.  In  this  case  the  target  is 
aligned  vertically,  which  produces  the  strongest  signal  and  is  the  most  favorable 
orientation  for  detection. 
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Sensor  Response  Curves 


•  Target-specific  signal  vs.  depth  and  orientation 


/  7 


Calculate  signal  vs. 
depth 


Physics  Basis 


We  then  calculate  how  the  response  varies  with  depth  using  simple  physics 
equations. 
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Sensor  Response  Curves 


•  Target-specific  signal  vs.  depth  and  orientation 


•  Weakest  signal  for  J-  \ 

horizontal  orientation  in^F  : 

0.0  0.5  1.0  1.5  2.0 

Depth  (m) 

Physics  Basis  10 


If  the  target  is  oriented  horizontally,  the  signal  is  weaker.  This  is  the  least  favorable 
orientation  for  detection. 
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Sensor  Response  Curves 


•  Target-specific  signal  vs.  depth  and  orientation 

/  7 


•  Calculated  signal  vs. 
depth  curve 


Physics  Basis 


The  variation  of  the  minimum  signal  vs.  depth  is  calculated  using  the  same 
equations. 
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Sensor  Response  Curves 


Target-specific  signal  vs.  depth  and  orientation 

/  7 


•  Signals  for  targets  at 
other  orientation  fall 
between  curves 

Physics  Basis 


Signals  for  targets  at  other  orientations  are  bounded  by  the  least  favorable  and 
most  favorable  curves. 
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Measured  Response 


•  Measured  signal  includes  response  to  target  plus  noise 

♦  Limits  detection  performance 

S  —  (J-qII  n-p  IqC  ^  -CTB  +  N 

V'  v - -V - 

measured  signal  from  target  noise 

response 

•  Noise  levels  can  vary  from  place  to  place  on  the  site  due 
to  variations  in  contributions  from  power  lines,  bouncing 
over  uneven  ground,  geology,  etc. 


Physics  Basis 
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Any  fluctuations  in  the  sensor  output  that  are  not  due  to  munitions  targets  or 
comparable  clutter  items  represent  noise  that  ultimately  limits  the  performance  of 
the  geophysical  system.  Noise  arises  from  a  variety  of  sources,  including  the  sensor 
electronics,  improper  or  careless  operation  of  the  sensor,  bouncing  and  jolting  over 
uneven  ground,  nearby  power  lines,  geology,  etc.  Some  of  these  factors  can  vary 
from  place  to  place  over  the  site  and  cause  significant  variations  in  the  noise  level. 
For  example,  a  part  of  the  site  where  the  ground  is  rougher  will  generally  have 
higher  survey  noise  because  of  increased  bouncing  and  heaving  of  the  sensor. 
Other  factors  that  can  introduce  noise  variability  from  place  to  place  include  power 
lines  running  through  the  site,  variations  in  the  underlying  geology,  varying  levels  of 
small  shrapnel  fragments,  etc. 
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Sensor  Response  Curve 


•  Noise  in  sensor  output  limits  detectability  of  deeper 
targets. 

♦  Signal  levels  5-6  times  the  RMS  noise  level  are  needed  for 


Physics  Basis  14 


Whether  or  not  a  given  object  will  be  detected  depends  not  only  on  how  strong  a 
signal  it  creates,  but  also  on  the  level  of  noise  in  the  measurements.  Reliable 
detection  requires  a  peak  signal  that  is  5-6  times  the  RMS  noise  level.  In  this  slide 
the  survey  noise  level  is  shown  by  the  dashed  line  on  the  response  curve  plot  from 
before.  Targets  with  signal  levels  below  a  few  tenths  of  a  millivolt  will  likely  be 
obscured  by  the  noise. 
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Response  Curve  Scaling 


Response  curves  for  different  munitions  items  all  have  the 
same  shape  but  are  scaled  by  the  specific  item's  response 
coefficients  (fL  and  (1T 
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Physics  Basis  0*0  0-5  1-0  1.5  2-0  15 

Depth  (m) 


The  response  curves  for  different  munitions  items  have  the  same  shape  but  are 
scaled  by  the  specific  item's  response  coefficients  ((3L,  (3T).ln  this  sequence,  we  start 
off  with  the  response  curves  for  a  60-mm  mortar. 

The  red  dots  show  measurements  of  the  EM61  (gate  2)  signal  from  a  60-mm  mortar 
at  various  depths  and  orientations.  The  lines  show  the  calculated  minimum  (least 
favorable  orientation)  and  maximum  (most  favorable)  signal-vs. -depth  response 
curves. 
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Response  Curve  Scaling 


Response  curves  for  different  munitions  items  all  have  the 
same  shape  but  are  scaled  by  the  specific  item's  response 
coefficients  (fL  and  (1T 
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Next  we  include  measured  signals  for  a  4.2-inch  mortar  (blue  dots) 
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Response  Curve  Scaling 


Response  curves  for  different  munitions  items  all  have  the 
same  shape  but  are  scaled  by  the  specific  item's  response 
coefficients  (fL  and  (1T 


Physics  Basis  0.0  0.5  1.0  1.5  2,0  15b 

Depth  (m) 


The  response  curves  simply  shift  up  or  down  to  bound  the  new  object's  response. 
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Measurement  Uncertainty 


Peak  signal  varies  with  true  (vs.  nominal)  target  depth  and  with 


target  location  relative  to  survey 


□istan  ce  from  C  enter  of  CoS  (cm ) 

Physics  Basis 


37  mm  signal  vs.  offset  from 
center  of  EM61  coil. 

♦  Dotted  lines  show  50  cm 
survey  line  spacing. 

♦  Signal  can  vary  by  15-20% 
depending  on  actual  target 
location. 

16 


Signal  strength  depends  very  strongly  on  the  distance  between  the  target  and  the 
sensor.  The  peak  signal  strength  measured  along  a  survey  line  over  a  target  will 
vary  depending  on  the  actual  vertical  distance  between  the  sensor  and  the  target 
(as  opposed  to  the  nominal  sensor  height  and  target  depth),  and  any  target  offset 
relative  to  the  survey  line.  For  a  target  that  is  nominally  20  cm  deep,  ±5  cm  vertical 
offset  due  to  sloppy  target  burial  or  sensor  bounce  can  produce  ±20%  variation  in 
the  measured  signal  strength.  The  plot  shows  the  measured  EM61  signal  for  a  37- 
mm  projectile  as  it  moves  across  the  sensor  at  fixed  vertical  separation.  The  EM61 
is  1  m  wide,  and  the  signal  strength  drops  to  about  14  its  peak  value  at  the  edges  of 
the  coil  (±50  cm  from  the  center).  Imagine  that  the  37-mm  projectile  is  in  a  grid  that 
is  being  surveyed  at  50-cm  line  spacing.  One  of  the  survey  lines  will  have  to  fall 
somewhere  between  the  dotted  lines  at  ±25  cm,  and  the  signal  from  the  37-mm 
projectile  could  vary  by  -15%  depending  on  where. 
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Response  Curve  Error  Bars 


Sensor  Influences  on  Data  Collection 


Legend 

Most  Favorable 
Orientation 
Least  Favorable 
Orientation 


Peak  Signal  (mV) 

0  1  10  100  1,000 


Physics  Basis 
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This  series  of  slides  illustrates  how  measurement  uncertainty  creeps  in.  As  noted  in 
the  previous  slide,  slight  variations  in  the  vertical  and  horizontal  offsets  between  the 
sensor  and  the  target  can  produce  significant  signal  variations.  Here,  the  sensor  is 
relatively  low  to  the  ground  (and  thus  close  to  the  target)  and  the  signal  is  relatively 
high. 
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Response  Curve  Error  Bars 


Sensor  Influences  on  Data  Collection 


Legend 

Most  Favorable 
Orientation 
Least  Favorable 
Orientation 


Peak  Signal  (mV) 

0  1  10  100  1,000 


Physics  Basis 
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Here  the  sensor  is  higher  above  the  ground  so  the  signal  is  relatively  low. 
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Response  Curve  Error  Bars 


Sensor  Influences  on  Data  Collection 
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Legend 
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We  represent  the  signal  uncertainty  that  arises  because  of  these  sensor  height 
variations  (or  equivalently  target  burial  depth  uncertainty)  by  error  bars  on  the 
response  curves. 
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Response  Curve  Error  Bars 


Sensor  Influences  on  Data  Collection 


Legend 

_  Most  Favorable 

Orientation 

Least  Favorable 

Orientation 

Sensor  Peak  Signal  (mV) 

o  1  10  100  1,000 


Physics  Basis 


17c 


For  each  of  the  three  target  positions,  there  is  an  error  bar  representing  sensor 
height  uncertainty  and  one  representing  measurement  noise. 


27 


Common  Munitions  Items 


Naval  Research  Laboratory 
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Response  curves  for  common 
munitions  items  published  in  NRL 
Report  NRL/MR/61 10-08-9155 

♦  37mm  projectile 

♦  60mm  mortar 

♦  75mm  projectile 

♦  81mm  mortar 

♦  4.2"  mortar 

♦  105mm  projectile 

♦  105mm  HEAT 

♦  155mm  projectile 

♦  3"  Stokes  Mortar 

♦  2.36"  Rocket 

♦  2.75"  Rocket 

♦  MK2  Hand  Grenade 

♦  M9  Rifle  Grenade 


Response  curves  for  common  munitions  have  been  published  by  the  Naval 
Research  Laboratory  in  report  NRL/MR/61 10-08-9155.  This  document  is  available 
in  the  SERDP/ESTCP  on-line  library,  which  may  be  accessed  at  www.serdp.org  or 
www.estcp.org. 
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Response  Curve  Calculator 


•  EM61  array  response  curves  for  common  munitions 

•  Single  sensor  response  curves  for  new  munitions  items 

♦  Requires  controlled  measurements  at  several  depths  for  scaling 

•  Distributed  with  GSV  Report 


Physics  Basis  19 


A  simple  stand-alone  computer  program  for  calculating  response  curves  for  arrays 
of  EM61  sensors  for  any  of  the  common  munitions  items  listed  on  the  previous 
slide,  or  calculating  single-sensor  EM61  response  curves  for  other  munitions  items 
is  available  with  the  Geophysical  System  Verification  (GSV)  report.  For  new  single¬ 
sensor  response  curves,  the  program  must  be  supplied  with  controlled  pit  or  test 
stand  measurements  of  the  response  at  several  sensor-to-target  ranges. 
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Industry  Standard  Objects 


Naval  Research  Laboratory 

Washington,  DC  20375-5320 


EM61-MK2  Response  of 
Three  Munitions  Surrogates 


Alexandria,  Virginia 


•  Readily  available,  similar  in  size  and 
shape  to  common  munitions  items 

•  Documented  response  curves 

•  Repeatable,  consistent  EM  signals  for 
calibration  and  performance 
validation 


Physics  Basis 


Item 

Nominal 
Pipe  Size 

Outside 

Diameter 

Length 

Part 

Number1 

as™ 

Specification 

Small  ISO 

1" 

1.315" 

(3  3  mm) 

4” 

(102  mm) 

44615K466 

A53/A773 

Medium 

ISO 

2" 

2.375" 

(60  mm) 

S11 

(204  mm) 

44615K529 

A53/A773 

Large  ISO 

4,r 

4.500" 

(115  mm) 

12" 

(306  mm) 

44615K137 

A53/A773 

1 .  Part  number  from  the  Me  Mas  ter- Carr  catalog. 


You  don't  need  to  measure  the  responses  from  munitions  items  to  verify 
geophysical  sensor  performance.  Using  this  physics-based  approach,  pretty  much 
anything  will  do.  We  recommend  test  items  that  are  readily  available,  inexpensive, 
and  similar  in  size  and  shape  to  common  munitions  items,  and  have  identified  a  set 
of  Industry  Standard  Objects  (ISOs)  with  documented  response  curves  which  can 
be  used  to  provide  repeatable,  consistent  EM  signals  for  sensor  calibration  and 
performance  validation.  The  ISOs,  shown  in  the  inset  picture,  are  standard  size 
steel  pipe  nipples  as  described  in  the  table.  The  small  ISO  is  roughly  the  size  of  a 
37-mm  projectile,  the  medium  size  ISO  is  comparable  to  a  60-mm  mortar,  and  the 
large  ISO  is  roughly  the  comparable  to  a  larger  munitions  item  like  105-mm 
projectile  or  4.2-inch  mortar.  They  will  produce  signals  that  are  similar  to  those  of 
the  corresponding  munitions  items. 
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SERDP 


Magnetics 


•  Signal  representation  for  total  field  magnetometers  is 
similar  to  EMI 

♦  Performance  can  be  accurately  calculated  using  basic  EM  theory 

•  EMI  GSV  apparatus  carries  over  directly  to  magnetics 

♦  Uncertainty  increased  by  effects  of  remnant  magnetization 


induced  dipole  moment 


magnetic  anomaly 

H  = 


Physics  Basis 
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Total  field  magnetometers  are  also  used  in  munitions  response  work.  They  are  kind 
of  similar  to  EMI  sensors,  but  simpler.  Most  importantly,  these  sensors  also  obey 
well-defined  basic  physical  principles,  and  their  performance  can  be  quantitatively 
characterized  with  a  high  degree  of  accuracy  and  precision.  The  signal 
representation  for  magnetics  is  similar  to,  but  much  simpler  than  for  EMI.  We  still 
use  a  far  field,  dipole  representation.  Munitions  items  and  comparable  clutter  are 
magnetized  by  the  earth's  field,  and  this  is  expressed  by  an  induced  magnetic 
dipole  moment.  The  strength  of  the  dipole  moment  is  directly  related  to  the  size  of 
the  object.  As  with  the  EMI  sensors,  signal  vs.  depth  response  curves  for 
magnetometers  can  be  calculated  using  basic  electromagnetic  theory.  The  major 
difference  is  that  the  uncertainty  (as  expressed  by  error  bars)  is  increased  due  to 
the  effects  of  remnant  magnetization. 
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Summary 


= SERDP 


•  Modern  geophysical  sensors  (mag  and  EM)  produce 
repeatable,  site  invariant  signals  which  can  be  accurately 
modeled  using  basic  electromagnetic  theory 

•  Signal  strength  vs.  depth  response  curves  can  be 
constructed  for  any  munitions  item  of  interest 

♦  Combined  with  knowledge  of  on-site  noise  levels,  response 
curves  can  be  used  to  set  anomaly  selection  criteria  or  evaluate 
expected  performance 

•  Responses  from  simple  standard  objects  (ISOs)  can  be 
used  as  surrogates  for  munitions  items  for  instrument 
calibration  and  performance  verification 


Physics  Basis 
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Modern  geophysical  sensors  (magnetometers  and  EMI  sensors)  produce 
repeatable,  calibrated  signals  which  can  be  accurately  modeled  using  basic 
electromagnetic  theory.  Consequently,  signal  strength  vs.  depth  response  curves 
can  be  constructed  for  any  munitions  item  of  interest.  Combined  with 
measurements  of  the  on-site  noise  levels,  the  response  curves  can  be  used  to  set 
anomaly  selection  criteria  or  evaluate  expected  performance.  Because  the 
responses  of  different  targets  scale  in  a  well-defined,  calculable  way,  the  responses 
from  simple  standardized  objects  (ISOs)  can  be  used  as  surrogates  for  munitions 
items  for  purposes  of  instrument  calibration  and  performance  verification. 
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Geophysical  System  Verification 

Jeff  Swanson 
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Geophysical  System  Verification 


•  Instrument  Verification  Strip 

♦  Verify  that  the  equipment  is  working  properly 

♦  Measure  site  noise  from  which  the  target  signal  must  be 
extracted 

•  Production  Blind  Seeding 

♦  Ongoing  monitoring  of  production  geophysics 


Concepts  2 


The  IVS  is  intended  to: 

•  verify  that  the  equipment  is  working  properly  and 

•  measure  site  noise  from  which  the  target  signal  must  be  extracted. 


The  Instrument  Verification  Strip 


•  Line  of  well-characterized  targets  and  noise  strip 

♦  Small  and  easy  to  construct 

♦  Convenient  and  representative  location(s)  on  the  site 

•  Data  collection  -  as  specified  for  production  survey 

♦  Prior  to  production  work 

♦  Twice  daily 

•  IVS  constructed,  data  collected  and  analyzed,  and  the 
results  reviewed  for  consensus  to  proceed  in  a  single  day 


Concepts  3 


The  IVS  envisioned  consists  of  a  line  of  well  characterized  objects,  preferably  ISOs, 
buried  in  an  area  representative  of  the  local  site  conditions.  Data  would  be 
collected  prior  to  beginning  production  work  using  the  same  protocols  specified  for 
the  field  data  collection.  The  first  day’s  IVS  survey  would  verify  that  DQOs  set  prior 
to  project  initiation  are  met  and  that  they  are  sufficient  to  meet  project  objectives. 

Then,  the  IVS  would  be  visited  twice  daily,  at  the  start  and  finish  of  the  field  work,  to 
verify  proper  sensor  operation.  Noise  will  be  measured  in  a  convenient  adjacent 
area.  It  is  envisioned  that  the  IVS  could  be  constructed,  data  collected  and 
analyzed,  and  the  results  reviewed  for  consensus  to  proceed  in  a  single  day. 
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TVS  Specification 


. /r\ . . . . 

\U 

•  Targets 

♦  Handful  of  objects  in  a  line 

♦  Industry  Standard  Objects  (ISOs)  recommended 

♦  Locations  known  to  sensor  operator 

•  Noise  strip 

♦  Representative  of  site  conditions 

♦  Sufficiently  far  from  objects 

Concepts  4 


The  contents  of  the  IVS  can  in  principle  consist  of  any  well  characterized  objects.  Our  preference  is 
to  use  the  ISOs,  for  which  sensor  response  curves  have  been  produced.  Strictly  speaking  only  one 
item  would  be  sufficient  to  provide  the  data  required  for  physics  based  confirmation  of  performance, 
that  is  to  ensure  that  the  sensor  system  is  recording  the  expected  signal  at  the  correct  location. 
Multiple  items  may  be  desired  to  provide  a  range  of  signals.  To  that,  a  project  team  could  add  inert 
versions  of  the  targets  of  interest.  The  IVS  outlined  here  is  centered  around  a  modest  number  of  the 
ISOs. 

The  items  will  be  buried  in  a  straight  row  and  are  not  intended  to  be  blind  to  the  sensor  operator.  To 
the  contrary,  the  lane  to  be  surveyed  should  be  well  marked,  so  that  the  sensor  will  pass  directly  over 
the  targets,  providing  an  accurate  measure  of  the  peak  signal.  The  distance  between  the  items 
should  be  sufficient  so  that  the  sensor  signal  level  returns  to  the  noise  level  between  the  test  strip 
items. 

Noise  measurements  will  be  made  on  an  adjacent  strip  containing  no  discrete  anomalies  or  non¬ 
representative  terrain  or  geology  that  will  affect  the  instrument.  To  be  most  convenient,  the 
background  noise  measurements  should  be  made  adjacent  to  or  within  the  IVS.  However,  noise 
measurements  should  made  be  far  enough  from  the  buried  targets  so  that  their  signals  do  not 
contaminate  the  measured  noise  background. 
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Object  Burial  Depth 


•  Objective  is  to  quantitatively 
verify  that  sensor  is  measuring 
correct  signal-  not  to  measure 
detection  depth 

•  Signal  can  be  verified  anywhere 
along  the  response  curve 

•  Requires  that  your  measurement 
is  primarily  signal  and  not  noise 

♦  Deeper  targets  with  low  SNR  difficult 
to  use  for  verification 

Concepts  5 


Depth  (cm)  with  Standard  40  cm  Lower  Coil  Height 
0  20  40  60  80 


The  objective  of  the  IVS  is  to  verify  correct  operation  of  the  sensor,  not  to  test  its 
maximum  performance,  which  we  can  calculate.  The  IVS  is  intended  to  provide 
quantitative  confirmation  that  the  sensor  is  achieving  the  performance  predicted 
from  the  sensor  response  curves.  A  few  measurements  at  any  point  along  the 
curve  are  sufficient  for  this  purpose,  and  high  SNR  measurements  are  preferred  for 
quantitative  comparisons.  Thus,  items  buried  in  the  IVS  should  be  at  depths  that 
provide  signals  well  above  the  sensor  noise  level  so  that  measurements  of  sensor 
signal  level  will  not  be  contaminated  by  significant  noise.  However,  because  of  the 
way  expected  signals  are  calculated,  the  targets  should  be  sufficiently  far  from  the 
sensor  that  the  dipole  approximation  is  valid.  Both  the  conditions  of  sufficient  signal 
and  dipole  response  can  be  met  for  burial  depths  of  3  to  7  times  the  target’s 
diameter.  For  example,  a  one-inch  ISO  buried  7  times  its  diameter  in  the  least 
favorable  orientation  provides  a  4.8  mV  signal  for  a  channel  2  of  an  EM61-MK2, 
which  would  provide  sufficient  signal  for  accurate  quantitative  measurements  in 
background  the  of  0.3  mV  RMS  (or  2  mV  peak-to-peak)  noise. 
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Why  are  ISOs  OK? 


Depth  (cm)  with  Standard  40  cm  Lower  Coil  Height 
0  20  40  60  80 


Concepts  Verify  one  curve,  verifies  all  curves  6 


Any  well-characterized  target  can  be  used  to  confirm  sensor  response  is  consistent 
with  predictions.  There  is  no  need  to  verify  each  curve  separately.  Here  are 
example  sensor  response  curves  for  three  small  munitions. 
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Why  are  ISOs  OK? 


Depth  (cm)  with  Standard  40  cm  Lower  Coil  Height 
0  20  40  60  80 
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40  60  80  100  120 

Distance  Below  Lower  Coil  (cm) 


Concepts  Verify  one  curve,  verifies  all  curves  6a 


The  small  ISO  can  verify  performance  in  the  region  relevant  to  similarly  sized 
munitions. 
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Initial  Data  Collection 


First  Day 


Concepts 
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The  IVS  is  meant  to  be  a  dynamic  test.  Data  should  be  collected  and  processed  in 
the  same  way  that  production  field  data  collection  will  be  done.  For  the  IVS  data  to 
be  representative,  adjustable  parameters  such  as  height  above  ground  and  survey 
speed  must  be  replicated,  but  just  as  importantly  the  data  processing  steps  should 
also  be  the  same. 

The  data  collected  prior  to  initiation  of  field  work  will  be  somewhat  more  extensive 
than  what  would  be  collected  daily.  At  a  minimum,  one  line  of  data  would  be 
collected  with  the  sensor  passing  directly  over  top  of  the  items.  This  will  provide  the 
peak  signal  measurements  to  confirm  sensor  operation. 

For  each  item  in  the  IVS,  the  peak  signal  strength  from  the  initial  day’s  data  should 
be  compared  to  the  expected  signal  for  consistency.  In  all  cases,  the  signal  should 
be  no  lower  than  the  predicted  signal  for  the  least  favorable  orientation,  and  only  the 
signals  for  objects  in  that  orientation  should  approach  the  lower  line.  The  location 
accuracy  should  be  verified  by  comparing  the  picked  signal  locations  to  the  known 
locations  of  the  buried  objects. 
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Initial  Data  Collection 


First  Day 


Instrument 

Verification 

Strip 
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A  second  pass  is  made  offset  from  the  targets  to  measure  site  survey  noise.  The 
noise  measured  here  has  been  plotted  on  the  sensor  response  curve. 
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Consensus  to  Proceed 


SERDP 


•  Confirm  design  specifications  will  meet  project  objectives 

♦  All  responses  as  predicted 

♦  Site  noise  allows  detection  of  targets  of  interest 

♦  Geolocation  is  within  specification 

•  Consensus  to  proceed  to  production  work 


Concepts 


Depth  (cm  bgs)  with  Standard  40  cm  Lower  Coil  Height 
0  20  40  60 


Distance  Below  Lower  Coil  (cm) 


o° 
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Requirements  for  consensus  to  proceed  from  the  first  day  on  the  IVS  to  field  work 
would  be  set  in  advance  and  consistent  with  the  data  quality  objectives.  The  intent 
of  the  first  day  is  to  confirm  that  the  design  specifications  are  met  and  that  they  are 
appropriate  to  meet  overall  project  objectives. 

All  signals  should  fall  within  the  two  bounding  sensor  response  curves  and 
geolocation  accuracy  within  specification.  Only  items  buried  at  their  least  favorable 
orientations  should  approach  the  lower  line.  The  RMS  noise  should  be  measured 
and  compared  to  the  expected  signal  values.  Together  the  signal  strength  and  the 
noise  levels  will  either  verify  that  the  targets  of  interest  can  be  detected  to  their 
depth  of  interest  or  inform  the  project  team  that  the  planned  data  collection  will  not 
meet  project  objectives. 
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Subsequent  Daily  Data  Collection 
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Concepts  9 


For  daily  instrument  checks,  data  would  be  collected  in  one  direction  down  the 
center  of  the  target  strip.  The  main  objective  of  the  daily  run  is  to  check  that  peak 
signal  levels  remain  consistently  as  predicted  and  that  the  system  noise  levels  have 
not  changed,  which  would  indicate  an  equipment  malfunction.  The  single  pass  will 
measure  geolocation  accuracy  only  in  one  direction,  which  will  still  be  sufficient  to 
identify  a  failure. 


11 


Subsequent  Daily  Data  Collection 
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Concepts  9a 


A  pass  is  also  collected  over  the  noise  strip  to  ensure  the  system  noise  is  consistent 
from  day  to  day. 
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Daily  Confirmation  of  System  Performance 

•  Monitor  system  performance  for  duration  of  project 

♦  Consistent  response  to  I  VS  targets 

♦  System  noise  remains  within  specification 

♦  Geolocation  remains  within  specification 
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Peak  signal  amplitudes  for  each  item  should  be  reproducible  from  day  to  day. 
Appropriate  acceptable  bounds  should  be  set,  considering  the  data  quality 
objectives,  as  well  as  the  contributions  of  various  error  sources.  Geolocation 
accuracy  should  be  within  specification. 
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Geophysical  System  Verification 


•  Instrument  Verification  Strip 

♦  Verify  that  the  equipment  is  working  properly 

♦  Measure  site  noise  from  which  the  target  signal  must  be 
extracted 

•  Production  Blind  Seeding 

♦  Ongoing  monitoring  of  production  geophysics 


Concepts  11 
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Concept 


Instrument 

Verification 

Strip 


Concepts 


•  Selecting  Seed  Targets 

•  Quantity,  Depth  and 
Orientation 

•  Emplacement 

Geophysical  System  Verification 


Blind  Seeds 


12 


The  seeding  program  envisioned  calls  for  the  placement  of  known  objects  at 
surveyed  locations  that  are  blind  to  the  survey  and  data  processing  teams  at 
sufficient  frequency  that  they  are  useful  for  daily  quality  checks.  At  a  minimum,  the 
seed  items  should  include  one  or  more  of  the  three  industry  standard  objects 
(ISOs).  The  main  purpose  of  the  seeds  is  to  provide  ongoing  verification  that  known 
objects  produce  signals  that  are  expected.  As  such,  they  need  not  mimic  the 
munitions  of  interest  in  every  detail.  The  ISO  that  is  selected  should  have  a 
signature  that  would  meet  the  anomaly  selection  criteria;  that  is,  you  should  expect 
that  the  seeds  would  be  selected  and  placed  on  the  dig  list.  Although  not  required 
by  the  GSV,  the  project  team  may  also  want  to  include  inert  versions  of  the 
munitions  expected  at  the  site  in  order  to  support  public  acceptance. 
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Selecting  Seed  Targets 


•  ISO  appropriate  to  munitions  of 
interest 

♦  Meet  anomaly  selection  criteria 

•  Based  on  most  stressing  munition 
on  the  site 

•  May  supplement  with  inert 
munitions 


•  Corner  stakes,  etc.  may  be  useful 
as  well,  but  do  not  qualify  as 
blind  seeds 

Concepts  13 


The  seed  program  should  be  built  around  the  ISO  that  is  appropriate  to  the 
munitions  of  interest  on  the  site.  In  most  cases,  the  ISO  should  meet  the  anomaly 
selection  criteria  and  have  a  signature  with  a  magnitude  and  spatial  extent  that  is 
close  to  the  object  of  interest.  On  most  sites,  there  will  be  more  than  one  munition 
type  of  interest.  In  this  case,  the  ISO  should  be  selected  based  upon  the  most 
stressing  target.  For  example,  the  site  team  might  pick  the  object  of  interest  with 
the  smallest  spatial  signature,  as  this  object  is  likely  to  drive  the  DQO  on  lane 
spacing. 

ISOs  may  be  supplemented  with  inert  munitions.  Although  this  is  not  required  for 
the  main  objective  of  the  seeding,  that  is  to  prove  that  the  detection  system  is 
working  and  that  the  targets  of  interest  will  be  detected  as  expected  from  prior  tests, 
the  use  of  some  inert  munitions  may  be  necessary  to  satisfy  the  public  or  aid  in 
communication. 

Other  common  field  procedures  maybe  exploited  to  augment  a  blind  seed  program. 
Corner  stakes  at  known  locations  can  be  used  to  verify  location  accuracy,  latency, 
consistency  of  response,  and  other  data  quality  measures.  Because  they  are  at 
locations  known  to  the  survey  and  data  processing  teams  and  they  appear  in  a 
regular  pattern,  they  would  not  be  regarded  as  acceptable  blind  seeds. 
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Quantity 

One  per  day  per  crew 

♦  Cart  ~  1  per  acre 

♦  Array  ~  1  per  10  acres 

Sufficient  frequency  to  be  a  useful  QC  tool 

More  in  the  grids  to  be  done  on  the  first  couple  days 


Instrument 
Verification  SCp 


Geophysical  System  Verification 
Prcd  ucfon  /tf&a 
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On  average,  at  least  one  seed  should  be  encountered  per  day  per  crew.  For  a  field 
crew  using  a  cart-based  EM-61,  the  daily  production  rate  might  be  1  acre.  One 
seed  per  acre  would  be  appropriate.  For  a  towed  array  system,  the  production  rate 
may  be  5-10  acres  per  day.  It  may  be  advantageous  to  place  a  higher  density  of 
seeds  in  the  lots  to  be  surveyed  in  the  first  few  days  of  production. 
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Depth  and  Orientation 


•  Depth 

♦  Depth  where  items  are  expected  to  be  detected 

♦  Can't  evaluate  non-detect  if  <100%  detection  expected 

•  Orientation 

♦  May  be  buried  in  variety  of  orientations 

♦  Horizontal  will  be  most  stressing 
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Concepts  Depth  (cm  bgs)  15 


Anomaly 

Selection 

Threshold 


Seeds  should  be  buried  at  depths  where  they  are  expected  to  be  detected.  It  would 
be  difficult  to  interpret  a  failure  if  items  were  buried  at  their  most  stressing  depths 
such  that  their  expected  probability  of  detection  was  not  100%.  Seeds  may  be 
buried  in  any  variety  of  orientations.  Quantitative  evaluation  will  be  most 
straightforward  for  seeds  buried  in  the  horizontal  (least  favorable)  orientation,  for 
which  the  sensor  response  curves  are  calculated. 
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Emplacement 


SERDP 


•  Prescribed  locations  notional 

♦  Anomaly  avoidance  may  necessitate  adjusting  locations 

-  Safety 

-  Clear  signal  to  interpret 

•  Actual  locations  critical 

♦  Northing  and  Easting  location 

-  Must  equal  or  exceed  accuracy  of  geolocation  system 

-  Based  on  certified  monument 

♦  Depth  to  center  of  object 

♦  Orientation 

•  Blind  to  data  collection  and  analysis  teams 


Concepts 
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Seeds  will  be  buried  in  the  production  site,  where  UXO  is  expected  to  be  present. 
The  seed  plan  will  specify  planned  locations  or  frequency  with  which  the  seeds  will 
be  encountered.  The  planned  locations  for  seeds  must  be  flexible  so  that  they  may 
be  emplaced  safely.  Anomaly  avoidance  should  be  practiced  in  the  burying  of 
seeds  and  all  procedures  should  be  in  compliance  with  relevant  safety  guidelines. 

As  long  as  the  actual  buried  location  is  accurately  recorded,  nothing  is  lost  by 
moving  a  planned  seed  by  a  few  meters. 

Seed  locations  should  be  surveyed  to  an  accuracy  that  equals  or  exceeds  the 
expected  accuracy  of  the  positioning  system.  Locations  should  be  based  on  a 
certified  monument  that  will  be  used  for  the  surveying.  Depth  to  a  specified  point 
should  be  recorded.  All  of  the  depths  in  this  report  and  in  Refs.  3  and  4  are  to  the 
center  of  the  object.  For  non-spherical  objects,  such  as  pipes  or  inert  munitions,  the 
orientation  of  the  object  in  the  ground  should  be  recorded. 

Evaluation  of  the  blind  seeds  could  in  principle  be  done  by  the  QCc  arm  of  the 
performer,  customer  or  an  independent  third  party.  What  is  important  is  to  maintain 
the  integrity  of  the  blind  seeds.  This  requires  that  the  truth  information  be 
segregated  from  the  people  collecting  and  processing  the  data,  as  well  as  those 
performing  the  target  picking. 
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Blind  Seed  Data  Collection 


Production  Blind  Seeds 


Concepts 
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Production  blind  seeds  will  be  encountered  by  the  survey  crew  in  each  grid  they 
survey. 
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Blind  Seed  Data  Collection 


Production  Blind  Seeds 
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The  data  required  from  the  data  analyst  include  information  that  is  commonly  found 
in  target  lists.  For  each  anomaly  that  meets  the  target  selection  criteria,  the 
analyst  should  report  at  a  minimum  the  peak  signal  strength  and  other  required 
parameters  and  the  X,Y  location. 

Signal  Strength.  A  plot  of  the  signal  strength  of  seed  targets  would  be  updated  as 
the  seeds  are  encountered.  The  expected  values  for  this  example  target  at  its 
most  and  least  favorable  orientations  are  shown  in  the  dashed  lines  and  the 
individual  measurements  are  represented  by  the  dots.  The  seeds  in  this 
example  are  buried  at  a  variety  of  depths  and  orientations. 

Location  accuracy:  A  polar  plot  of  the  location  accuracy  for  seeds  depicts  the  offset 
of  each  seed  location  if  the  target  were  centered  at  (0,0).  This  also  would  be 
updated  daily  as  the  seeds  are  encountered. 
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Production  Blind  Seed  Evaluation 

•  Ongoing  monitoring  of  data  collection,  processing  and 
target  selection  in  the  production  work 

♦  All  seeds  found 

♦  Peak  responses  within  the  expect  range 

♦  Location  within  tolerance 


All  signals  should  fall  within  the  two  bounding  sensor  response  curves.  Only  items 
buried  at  their  least  favorable  orientations  should  approach  the  lower  line.  In  this 
example,  the  seeds  were  buried  at  three  depths  and  a  variety  of  orientations.  The 
RMS  noise  should  be  measured  and  compared  to  the  expected  signal  values. 
Together  the  signal  strength  and  the  noise  levels  will  either  verify  that  the  targets  of 
interest  can  be  detected  to  their  depth  of  interest  or  inform  the  project  team  that  the 
planned  data  collection  will  not  meet  project  objectives. 
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Application  to  an  Example  Site 


Andy  Schwartz 
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SERDP 


Example  Site 

•  100-hectare  (250-acre)  site 

♦  Part  of  a  former  bombing  and  gunnery  range 

♦  Slated  for  residential  development 

•  A  variety  of  munitions  were  used  on  the  site 

♦  Primary  concern  is  37-mm  projectiles 

•  EM61-MK2  chosen  as  the  geophysical  sensor 

♦  targets  of  interest  are  relatively  small 

♦  not  expected  to  be  more  than  one  foot  deep 

♦  survey  lane  spacing  of  0.6  m 

•  Open  field  with  good  sky  view  throughout 

♦  GPS  system  is  the  choice  for  sensor  location 


Application  to  an  Example  Site 
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The  example  site  to  be  considered  is  a  100-hectare  (250-acre)  site  that  is  part  of  a 
former  bombing  and  gunnery  range.  After  remediation,  the  site  is  slated  for 
residential  development.  The  historical  records  indicate  a  variety  of  munitions  were 
used  on  the  site  but  the  primary  concern  is  37-mm  projectiles.  Since  the  targets  of 
interest  are  relatively  small  and  are  not  expected  to  be  more  than  one  foot  deep,  the 
site  team  has  chosen  the  EM61-MK2  as  the  geophysical  survey  instrument  to  be 
used  at  this  site  with  a  survey  lane  spacing  of  0.6  m.  The  site  is  an  open  field  with 
good  sky  view  throughout  so  a  GPS  system  is  the  choice  for  sensor  location. 
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Summary  of  Project  Data  Objectives 


Parameter 

Objective  at  this  Site 

Geophysical  Instrument 

EM61-MK2  on  standard  wheels 

Geolocation  System 

RTK  GPS  (cm-level) 

Depth  of  Interest 

30  cm  (1  foot) 

Survey  Lane  Spacing 

60  cm 

Gate  for  Primary  Data  Analysis 

Gate  2 

IVS  Anomaly  Amplitude 
Reproducibility 

±  20% 

IVS  Item  Position  Reproducibility 

±  25  cm 

Location  Accuracy  of  Seed  Picks 

±  50  cm 

Application  to  an  Example  Site 
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This  table  summarize  the  decisions  made  by  the  site  team.  Notice  that  position 
reproducibility  specification  is  tighter  for  the  IVS  items  than  for  the  seeds.  The 
survey  team  should  be  able  to  center  the  sensor  directly  over  the  line  of  IVS  items 
so  the  position  reproducibility  should  be  high. 
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Establishing  the  FVS 


•  Choose  an  area  on  the 
site  that  contains 
representative  terrain, 
geology,  and  vegetation 

•  Make  sure  the  area 
chosen  is  free  of 
anomalies 

•  Bury  the  I  VS  items, 
recording  the  location 
and  depth  to  a  precision 
of  ±  2cm 

Application  to  an  Example  Site 


4 


The  expected  time  line  for  the  first  use  of  the  strip  involves  the  geophysical 
contractor  arriving  on  the  site  the  first  day  of  operations,  identifying  a  location  for  the 
test  strip  in  conjunction  with  the  program  manager,  conducting  a  background  survey 
to  identify  a  site  suitable  for  a  test  strip,  and  emplacing  the  test  items  according  to 
the  specification  in  the  previous  slide.  If  the  test  strip  location  is  not  very  cluttered, 
this  may  still  leave  time  for  an  initial  survey  on  the  first  day  at  the  site;  if  not,  the  test 
strip  can  be  surveyed  at  the  beginning  of  the  second  day  on  site. 
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Instrument  Verification  Strip 


30  m  - 1 


1 

<=> 

.= 

5.0  m 

Item 

ID 

Position 

(m) 

Depth  to 
Center 

Orientation 

Predicted 

Response 

1 

Small  ISO 

2.5 

11  cm  (3X) 

Horiz.  across  track 

11.6  mV 

2 

Small  ISO 

7.5 

26  cm  (7X) 

Horiz.  across  track 

4.8  mV 

3 

37-mm  Proj. 

12.5 

11  cm 

Horiz.  across  track 

18.3  mV 

4 

37-mm  Proj. 

17.5 

26  cm 

Horiz.  along  track 

6.7  mV 

5 

Small  ISO 

22.5 

11  cm 

Horiz.  along  track 

11.6  mV 

6 

Small  ISO 

27.5 

26  cm 

Horiz.  along  track 

4.8  mV 
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The  first  task  in  planning  the  instrument  verification  strip  is  to  decide  what  items  will 
be  emplaced.  The  site  team  at  this  site  decided  that  since  the  smallest,  most 
difficult  to  detect,  item  of  interest  is  a  37-mm  projectile,  the  test  strip  would  contain 
two  inert  37-mm  projectiles  and  four  small  Industry  Standard  Objects  (ISOs)  to 
serve  as  surrogates  during  the  seed  program.  They  will  be  placed  in  the  IVS  at  two 
depths  (3X  and  7X  their  diameter)  and  two  orientations.  Note  that  the  deepest 
depth  chosen  is  close  to  the  maximum  depth  of  interest  at  this  site  but  that  was  not 
the  reason  for  the  choice.  The  goal  of  the  IVS  is  to  verify  twice  each  day  that  the 
geophysical  system  is  working  correctly.  To  accomplish  that  with  reasonable 
precision  requires  a  high  SNR  on  the  sensor  measurements.  The  two  depths  were 
chosen  to  ensure  the  required  SNR  is  achieved. 

The  items  to  be  emplaced  are  relatively  small  so  the  spatial  extent  of  their 
signatures  will  not  be  large  but  an  ancillary  purpose  of  the  test  strip  is  to  get  a 
measure  of  site-specific  survey  noise.  With  this  in  mind,  the  site  team  decided  to 
emplace  the  test  strip  items  with  spacing  of  5  meters,  leaving  2.5  meters  clear  on 
each  end  of  the  strip.  This  results  in  a  test  strip  approximately  100  feet  long. 
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First  Day’s  Data  Collection 


0.6  m 


Confirm  Line  Spacing 
Verify  Response 
Verify  Positioning 
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The  protocol  for  the  first  day’s  survey  of  the  IVS.  The  first  pass  of  the  1-m  wide  by 
%  m  EM61-MK2  is  made  with  the  sensor  0.6  m  offset  from  the  test  item  burial  line. 
The  site  team  has  determined  a  line  spacing  of  0.6  m  is  appropriate  to  ensure 
detection  of  the  37-mm  projectiles.  The  next  pass  is  directly  over  the  test  items. 
This  will  allow  the  data  analyst  to  determine  the  maximum  signal  expected  from 
each  item.  The  third  pass  is  at  an  offset  of  0.6  meter  on  the  other  side  of  the  line  of 
items. 
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First  Day’s  Data  Collection 


h 


30  m 


H 


2.0  m 


•  Confirm  Line  Spacing 

•  Verify  Response 

•  Verify  Positioning 

•  Measure  Survey  Noise 
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The  protocol  for  the  first  day’s  survey  of  the  IVS.  The  final  pass  is  two  meters  offset 
from  the  line  of  targets  to  make  a  measurement  of  survey  noise  at  this  location. 
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Profile  Directly  Over  the  Items 
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A  trace  of  the  measured  data  from  the  line  directly  over  the  targets.  As  in  all  cases 
in  this  example,  these  are  actual  field  data  measured  over  an  IVS  constructed  as 
specified  above. 

Each  of  the  items  is  detected  with  good  SNR  and  the  teams’  choice  of  5  m  spacing 
between  the  items  is  confirmed;  each  anomaly  returns  to  the  baseline  and  there  is  a 
good  section  to  measure  noise  between  the  anomalies.  Notice  that  items  4,  5,  and 
6  display  the  familiar  double-humped  profile  that  is  the  signature  of  long  targets 
aligned  along  track.  Items  5  and  6  are  the  symmetric  Industry  Standard  Objects 
and  exhibit  a  very  clean  double-peaked  profile.  Item  4,  the  37-mm  projectile,  is  less 
symmetric  (the  nose  is  much  smaller  than  the  back  of  the  projectile)  with  the 
response  from  the  back  larger  than  that  from  the  nose  making  it  more  difficult  to 
establish  the  location  of  the  center  dip. 
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Profile  From  the  Noise  Measurement 
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A  trace  of  the  measured  data  from  the  “noise”  line  plotted  on  the  same  scale  as  the 
signal  trace  with  an  inset  at  a  higher  magnification.  There  may  be  a  small  scrap  item 
remaining  about  4.5  m  down  the  line  but,  otherwise,  the  contractor  team  has  done  a 
good  job  identifying  a  target-free  area  for  the  noise  measurements.  The  RMS 
survey  noise  in  this  area  is  0.25  mV  or  about  1.5  mV  peak-to-peak. 
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Are  the  Items  of  Interest  Detectable? 
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The  measured  anomaly  amplitude  in  Gate  2  for  the  two  37-mm  projectiles  and  the 
RMS  noise  are  compared  to  the  predicted  response.  The  blue  curve  corresponds 
to  the  signal  expected  when  the  item  is  in  its  most  favorable  (vertical)  orientation 
and  the  red  curve  corresponds  to  expected  signal  when  the  item  is  in  its  least 
favorable  (horizontal)  orientation.  Both  of  the  37-mm  projectiles  in  the  IVS  are 
oriented  horizontally  so  their  signals  should  be  close  to  the  solid  curve  if  the  sensor 
is  operating  normally,  which  it  is  in  this  case. 
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Are  the  Items  of  Interest  Detectable? 
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The  error  bars  on  the  measured  points  correspond  to  ±  5  cm  depth  uncertainty 
(horizontal)  and  twice  the  RMS  noise  (vertical). 
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Are  the  Items  of  Interest  Detectable? 


Depth  (cm  bgs) 
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•  Min.  signal  at  30  cm  =  5.2  mV 

•  Threshold  =  2.6  mV 

•  RNS  noise  =  0.25  mV 

•  SNR  >  20 
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From  the  site  noise  data,  the  site  team  can  confirm  that  the  detection  requirements 
for  this  item  at  this  site  can  be  met.  The  depth  of  interest  for  the  37-mm  is  1  foot  or 
~30  cm.  The  minimum  signal  in  gate  2  expected  from  a  37-mm  projectile  at  this 
depth  is  a  little  over  5  mV.  The  measured  survey  noise  in  this  gate  at  this  site  is 
0.25  mV  resulting  in  a  minimum  signal-to-noise  ratio  of  almost  20  which  is  well 
above  the  requirements  for  detection  and  should  lead  to  few,  if  any,  noise  picks. 
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t SERDP 

Geolocation  Performance  from 
Profile 


The  second  objective  to  be  checked  from  the  first  day’s  data  is  the  performance  of 
the  sensor  geolocation  system.  One  method  to  accomplish  this  is  to  find  the 
position  of  the  peak  signal  for  each  object  (or  in  the  case  of  targets  located  along 
track,  the  center  of  the  double-humped  profile)  and  compare  this  to  the  known 
locations  of  the  targets.  Since  the  GPS  system  used  at  this  site  measures  the 
position  of  the  center  of  the  EM61  coil,  this  cross-track  location  accuracy  is  limited 
by  how  carefully  the  senor  operator  positions  the  center  of  the  coil  directly  over  the 
line  of  items  in  the  IVS.  In  this  case  the  operator  was  very  careful,  resulting  in  the 
measured  position  deviations  shown  here  in  a  polar  plot.  The  IVS  at  this  site  is  laid 
out  E-W  so,  as  expected,  the  greatest  deviations  are  in  the  cross-track  (N-S) 
direction.  Had  any  of  the  deviations  been  larger  than  the  objective  of  25  cm, 
corrective  action  would  have  been  required  before  consensus  to  proceed  was 
achieved. 
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Interpolated  Image  from  Three  Traverses 
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A  better  method  to  determine  the  geolocation  performance  is  to  take  advantage  of 
the  extra  survey  passes  on  the  first  day  and  invert  the  geophysical  data  for  the 
item’s  position.  The  data  corresponding  to  the  passes  at  offsets  of -60,  0,  and  60 
cm  are  presented  here.  Each  anomaly  in  the  data  is  selected  by  the  data  analyst 
using  that  contractor’s  standard  methodology  and,  at  a  minimum,  analyzed  for 
location. 
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Results  of  First  Day’s  TVS  Survey 

•  Data  collected  on  the  I  VS 

♦  Confirm  sensor  selection 

♦  Confirm  that  targets  of  interest  are  detectable  to  the  depth  of 
interest 

♦  Demonstrate  the  geophysical  sensor  and  geolocation  equipment 
are  operating  correctly 

•  All  site  team  members  agree  so  consensus  is  achieved  to 
proceed  to  production  work 
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The  IVS  provides  a  simple,  but  rigorous,  verification  that  the  geophysical  mapping 
system  (sensor  plus  geolocation  equipment)  is  operating  properly.  From  the  data 
collected  on  the  first  day,  the  site  team  is  able  to  agree  that  the  correct  sensor  has 
been  chosen,  that  the  targets  of  interest  are  detectable  to  the  depth  of  interest  in  the 
presence  of  the  measured  survey  noise,  and  that  the  data  are  being  collected 
correctly. 

Given  these  results,  achieving  consensus  to  proceed  is  straightforward. 
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Twice  Daily  Data  Collection 


2.0  m 


•  Verify  Sensor  Response  and  Geolocation  Performance 

•  Confirm  Survey  Noise 
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In  addition  to  whatever  function  tests  the  contractor  performs  each  day  to  ensure 
proper  operation  of  their  survey  equipment,  each  survey  crew  will  be  required  to 
survey  the  test  strip  at  the  beginning  and  end  of  each  day.  This  will  be  a  simplified 
survey  as  illustrated  here,  one  pass  over  the  line  of  emplaced  targets  to  confirm 
sensor  operation  and  one  pass  to  confirm  that  the  survey  noise  has  not  changed.  If 
the  sensor  performance  and  system  noise  are  within  specifications  before  and  after 
each  day  of  surveying,  it  is  reasonable  to  expect  that  the  system  was  performing 
within  acceptable  bounds  throughout  the  day.  If  the  sensor  performance  is  within 
performance  criteria  in  the  morning  and  not  in  the  evening,  the  data  must  be 
examined  to  determine  if  any  of  it  is  usable. 

The  results  of  these  twice-daily  performance  confirmation  surveys  will  be  reported  in 
a  continually-updated  set  of  plots  showing  the  down-track  position  error  and 
amplitude  variation  for  each  target.  As  with  the  first  day’s  measurements,  any 
deviations  outside  of  the  data  objectives  will  require  a  detailed  failure  analysis 
before  survey  operations  can  be  resumed. 
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Plots  of  the  down-track  location  of  the  measured  anomalies  corresponding  to  the 
IVS  items.  The  points  correspond  to  the  locations  determined  each  morning  and 
evening  and  the  dashed  dines  correspond  to  the  ±  25  cm  specification  for  this 
measurement. 


17 


Notice  in  that  the  measured  down-track  position  of  Item  4  appears  to  have  an  offset 
from  the  known  value.  This  arises  from  the  difficulty  in  determining  the  center  of  the 
signal  for  this  item  and  illustrates  the  advantages  of  the  standard  targets  for  the  IVS. 
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Twice  daily  variation  in  the  measured  anomaly  amplitude  for  two  of  the  IVS  items 
Similar  plots  corresponding  to  all  six  items  are  updated  daily.  The  points 
correspond  to  the  measured  anomaly  amplitudes  and  the  dashed  lines  represent 
20%  of  the  mean  for  each  item. 


Production  Blind  Seeding 


•  The  site  team  has  hired  a  third  party  to  design  and 
implement  a  blind  seed  program  in  the  production 
survey  areas 

•  Only  one  component  of  QC  plan  for  the  site 

♦  Verifies  geolocation,  sensor  performance,  anomaly  selection,  and 
anomaly  resolution  on  an  on-going  basis 

•  Based  on  the  small  ISO 

♦  Surrogate  for  37-mm  projectile 
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Seed  Plan 


•  Site  has  been  divided  into  50  m  x  50  m  grids 

♦  400  grids 

•  Seeds  will  be  emplaced  in  all  the  grids 

♦  1/3  at  10  cm 

♦  1/3  at  20  cm 

♦  1/3  at  30  cm 

♦  Random  orientations 

•  Three  seeds  in  the  first  grid  to  be  surveyed  by  each  team 

•  25  of  the  shallow  seeds  will  have  another  seed  placed 
below  it 
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The  site  has  been  divided  into  50  m  x  50  m  grids.  Under  the  conditions  at  this  site, 
each  survey  team  covers  one  grid  per  day.  The  site  team  has  determined  that  to 
adequately  measure  the  performance  of  each  team,  they  will  require  a  seed  in  each 
of  the  grids  in  addition  to  any  seeding  the  contractor  employs  for  their  own  quality 
program.  This  means  -400  seeds  items  will  be  required.  One  third  of  the  seeds  will 
be  placed  at  10  cm,  one  third  at  20  cm,  and  one  third  at  our  depth  of  interest,  30 
cm,  in  random  orientations  with  all  measurements  corresponding  to  the  center  of 
the  item.  Twenty  five  of  the  shallow  seeds  will  have  an  additional  seed  placed 
under  them  for  the  purposes  of  confirming  the  anomaly  resolution  process  (stacked 
seeds).  In  addition  to  this,  three  seeds  will  be  placed  in  the  first  grid  surveyed  by 
each  of  our  three  survey  crews. 
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Evaluation  of  Performance 


•  Data  from  each  grid  and  anomalies  selected  transmitted  to 
consulting  geophysicist 

•  For  each  grid  that  contains  a  seed,  she  will  determine 

♦  Was  the  seed  selected  as  a  target 

♦  Are  the  signal  strength  and  target  location  within  limits 

•  If  the  seed  is  not  on  the  target  list 

♦  Is  there  a  signal  at  the  seed  location  that  should  have  been  picked? 

♦  Is  there  an  anomaly  but  it  is  below  threshold? 

♦  Is  there  a  problem  with  the  geolocation  system? 

•  After  dig  check 

♦  If  the  correct  item  (or  items  for  the  stacked  seeds)  is  recovered 
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Performance  evaluation  against  the  seeds  can,  in  principle,  be  done  by  the 
performer  or  a  third  party  employed  for  this  purpose.  The  only  requirement  is  the 
seeds  be  blind  to  the  personnel  collecting  the  data,  analyzing  the  data,  and 
selecting  targets  for  the  dig  list.  At  this  site,  a  consultant  geophysicist  has  been 
hired  to  oversee  the  blind  seeding  program  and  the  performer  has  chosen  to  plant 
some  additional,  non-blind  seeds  for  their  own  quality  program. 

As  the  data  from  each  grid  are  analyzed  and  targets  selected,  this  information  will 
be  transmitted  to  the  consulting  geophysicist.  For  each  grid  that  contains  a  seed, 
she  will  determine  whether  the  seed(s)  made  it  to  the  target  list.  If  it  did,  she  will 
ensure  the  signal  strength  and  location  accuracy  are  within  contract  specifications 
and,  after  the  anomaly  has  been  dug,  make  sure  that  the  correct  item  (or  items  if 
this  was  a  stacked  seed)  is  recovered.  If  the  seed  is  not  on  the  target  list,  she  will 
begin  a  root  cause  analysis.  Questions  to  be  asked  include:  is  there  a  geophysical 
signal  at  the  seed  location  that  should  have  been  picked?;  is  there  an  anomaly  but 
is  it  below  the  selection  threshold?;  is  there  an  anomaly  remaining  that  was  below  a 
more  shallow  anomaly  (stacked  seed)?;  and  is  there  a  sensor  location  issue? 
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One  of  the  products  of  the  performance  analysis  is  shown  here.  Just  as  for  the  IVS, 
the  geophysicist  checks  to  make  sure  the  anomaly  amplitude  measured  for  each 
seed  is  within  the  expected  bounds.  Since  the  seeds  were  buried  with  random 
orientations,  the  measured  amplitudes  are  expected  to  span  the  signal  between  the 
least-  and  most-favorable  orientations. 

The  error  bars  on  the  measured  amplitudes  correspond  to  ±  2.5  cm  in  depth  and 
twice  the  measured  site  noise. 
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Evaluation  of  Performance 
Location  Accuracy 


Analysis  of  the  seed  location  data  shows  that  a  bias  to  the  west  is  beginning  to  be 
evident.  Although  the  performance  still  meets  the  DQOs,  it  would  be  wise  to  begin 
to  investigate  the  cause  of  this  bias. 


24 


~  SERDP 


ESTCP 


Evaluation  of  Performance 
Missed  Seed 
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Geophysical  data  from  an  area  of  the  site  where  a  seed  was  missed.  The  X’s 
represent  targets  that  appear  on  the  pick  list,  and  the  circle  denotes  the  missed 
seed.  In  this  case,  a  response  is  present  at  the  location,  but  it  was  not  picked  in  the 
analysis  process.  A  root  cause  analysis  would  be  initiated  to  identify  the  failure  and, 
if  necessary,  prescribe  a  corrective  action. 
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Missed  Seed 
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In  this  case,  it  was  found  that  the  missed  seed  was  right  on  the  boundary  of  the 
grids  established  by  the  contractor  to  facilitate  their  survey  but  was  chosen  in 
neither.  A  procedure  was  establish  to  choose  all  anomalies  on  the  boundary  in  both 
grids  and  then  deal  with  the  redundancy  when  the  final  list  is  compiled. 
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Project  Summary 


•  IVS  constructed  and  surveyed 

♦  Consensus  to  proceed  on  the  same  day 

•  Daily  IVS  verified  continued  system  performance 

•  Production  blind  seeds 

♦  Detected  a  flaw  in  the  data  processing  approach 

♦  Production  results  verified  design  criteria  were  met 

•  Coupled  with  other  project  QC  built  a  strong  and 
defensible  case  that  if  munitions  were  on  our  site  and  we 
went  over  them,  they  were  detected  and  recovered 
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Moving  Beyond  GPOs 


•  Original  GPO  objectives  not  abandoned 

♦  "Quantitatively  understand,  measure  and  describe  the 
effectiveness  of  UXO  detection  technology" 

•  Geophysical  System  Verification 

♦  Builds  on  a  wealth  of  accumulated  understanding 

♦  Physics-based 

♦  Provides  continual  monitoring  for  the  life  of  the  project 

•  GSV  complements  existing  QA/QC 
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The  GSV  process  outlined  here  is  both  straightforward  and  rigorous  and,  combined 
with  other  quality  measures,  meets  the  historical  objectives  of  the  GPO.  It  redirects 
resources  from  a  traditional  GPO  to  a  quantitative  and  transparent  evaluation  of 
data  quality  that  spans  the  life  of  the  project. 

Daily  visits  to  the  IVS  before  and  after  the  production  work  will  provide  quantitative 
checks  on  signal  and  noise  consistency  in  a  known  location.  Blind  seeds,  emplaced 
at  intervals  to  provide  a  minimum  of  one  seed  in  each  day’s  data  collection,  provide 
ongoing  evidence  that  the  entire  data  collection  and  analysis  process  is  working. 
Seeds  should  be  detected  at  correct  signal  levels  and  placed  on  the  dig  list. 
Monitoring  of  noise  in  production  data  ensures  that  either  expectations  of  detection 
performance  will  be  met  or  that  changes  in  noise  conditions  are  recognized  and 
appropriate  adjustments  made.  Taken  together,  these  elements  of  the  GSV  lead  to 
confidence  that  data  throughout  the  project  is  meeting  project  objectives. 
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Added  Advantages  of  GSV 


•  Better  use  of  resources 

•  One  mobilization 

•  Offers  standardization  across  project  sites 

•  Science-based  decision  making 

•  Puts  added  resources  toward  project  objectives  rather 
than  document  production 

en  it  eooB  covernment 


Wrap  Up 
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GSV  is  a  more  efficient  use  to  limited  resources. 

One  deployment.  The  current  GPO  is  expensive,  in  part  because  it  often  involves  one 
deployment  to  construct  the  GPO  and  collect  the  data,  followed  by  an  interval  in  which  a 
formal  report  is  written,  reviewed  and  approved.  This  is  not  necessary.  With  advanced 
planning  based  on  an  understanding  of  the  geophysical  equipment,  the  essential  items 
could  be  verified  on-site  in  a  timely  manner  to  support  a  single  deployment.  Conditions 
must  be  outlined  in  detail  in  advance,  including  metrics  and  success  criteria.  If  GSV  is 
completed  successfully,  approval  to  proceed  could  be  immediate.  If  problems  are 
encountered,  the  information  generated  in  the  IVS,  noise  measurements  and  evaluation  of 
initial  data  collection  would  be  useful  for  failure  analysis  and  corrective  action. 

Can  be  standard  across  sites.  GPOs  have  always  varied  considerably,  in  size,  number 
and  selection  of  targets,  and  evaluation  criteria.  This  has  made  it  difficult  to  compare  data 
from  one  GPO  to  another.  Whether  the  targets  were  inert  munitions  or  other  objects  like 
pipes  or  spheres,  none  was  ever  well  characterized  and  did  not  support  any  type  of 
quantitative  interpretation.  Using  the  response  curves  generated  for  both  the  proposed 
ISOs  and  common  munitions,  it  is  now  possible  to  establish  a  minimum  of  standardization 
across  sites,  while  recognizing  that  individual  site  teams  may  want  to  add  to  what  is 
presented  here. 

Science-Based  Decision  Making.  The  GSV  allows  a  project  team  to  set  quantitative  criteria 
defining  acceptable  data  for  a  project  to  proceed.  Recognizing  how  the  site-invariant  signal 
of  an  object  of  interest  and  site-specific  noise  combine  to  determine  detecability  of  targets 
sets  realistic  expectations  for  meeting  project  objectives.  With  proper  QC  of  the  production 
data,  the  GSV  enhances  confidence  that  objectives  are  met  throughout  the  project. 
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